The duality between information embedding and source coding with side information and some applications

Aspects of the duality between the information-embedding problem and the Wyner–Ziv problem of source coding with side information at the decoder are developed and used to establish a spectrum new results on these and related problems, with implications for a number of important applications. The singleletter characterization of the information-embedding problem is developed and related to the corresponding characterization of the Wyner–Ziv problem, both of which correspond to optimization of a common mutual information difference. Dual variables and dual Markov conditions are identified, along with the dual role of noise and distortion in the two problems. For a Gaussian context with quadratic distortion metric, a geometric interpretation of the duality is developed. From such insights, we develop a capacity-achieving information-embedding system based on nested lattices. We show the resulting encoder–decoder has precisely the same decoder–encoder structure as the corresponding Wyner–Ziv system based on nested lattices that achieves the rate-distortion limit. For a binary context with Hamming distortion metric, the information-embedding capacity is developed, along with its relationship to the corresponding Wyner–Ziv rate-distortion function. In turn, an information-embedding system for this case based on nested linear codes is constructed having an encoder–decoder that is identical to the decoder–encoder structure for the corresponding system that achieves the Wyner–Ziv rate-distortion limit. Finally, based on these results, a simple layered joint source–channel coding system is developed with a perfectly symmetric encoder–decoder structure. Its application and performance is discussed in a broadcast setting in which there is a need to control the fidelity experienced by different receivers. Among other results, we show that such systems and their multilayer extensions retain attractive optimality properties in the Gaussian-quadratic case, but not in the binary-Hamming case.